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COMPUTER  GENERATED  IMAGE:  RELATIVE 
TRAINING  EFFECTIVENESS  OF  DAY 
VERSUS  NIGHT  VISUAL  SCENES 


L  INTRODUCTION 


The  Air  Force  is  in  the  process  of  increasing  its  simulator  training  capabilities  for  most  weapons 
systems  to  include  eonlact/visual  flight  training.  In  some  cases,  entirely  new  simulators  are  being 
designed;  in  other  cases,  existing  devices  are  being  improved.  In  either  case,  an  important  area  of 
consideration  is  the  choice  of  a  visual  display  system.  N  umerous  options  for  simulator  visual  systems 
are  commercially  available.  Major  system  types  include  TV  Model  Board,  Computer-Generated 
Image  (CGI),  and  Point  Light  Source  systems.  Across  and  within  each  system  type,  a  variety  of 
choices  can  be  made  with  respect  to  acquisition  costs  and  the  ability  to  meet  training  objectives. 
Although  information  is  available  to  specify  system  costs,  little  information  is  available  to  determine 
training  benefits. 

The  research  was  undertaken  to  determine  the  extent  to  which  takeofManding  skills  learned  in  a 
sintultor  equipped  with  a  night  visual  system  would  transfer  to  daytime  performance  in  the  aircraft. 
The  possibility  that  learning  to  fly  with  the  aid  of  a  night  visual  scene  simulator  would  result  in  flying 
habits  and  strategies  specific  to  nighttime  flying  raises  questions  regarding  the  extent  and  direction  of 
training  transfer  that  can  be  expected  for  daytime  flight  training.  An  area  of  particular  concern  is  in 
the  critical  stages  of  flight  where  any  negative  transfer  might  be  particularly  hazardous,  especially  for 
the  novice  pilot.  However,  if  substantia]  positive  transfer  could  be  achieved  using  a  night  scene  (at 
least  comparable  to  other  types  of  displays),  considerable  cost  savings  could  be  realized.  Training 
effectiveness  data  are  needed  in  order  to  supply  information  required  in  a  cost-to-benefit  analysis 
model,  particularly  when  the  choice  is  between  an  add-on  or  retrofit  modification  to  an  existing 
device  versus  procurement  of  an  entirely  new  device. 

Background 

Additional  hazards  are  associated  with  night  flying  and  have  been  well  documented  in  the 
aviation  literature.  In  addition  to  vertigo  and  spatial  disorientation,  poor  final  approaches  have  a 
higher  probability  of  occurrence  at  night.  In  a  survey  conducted  for  the  Navy.  Simons  (1965) 
reported  a  day/night  carrier  landing  accident  ratio  of  1:4.  In  subsequent  research  on  day/night 
carrier  approaches,  Brictson  (1966)  reported  that  pilot  tended  to  fly  lower  approaches  and  to  land 
longer  and  softer  at  night  than  by  day.  The  most  revealing  measures  were  those  of  variability,  with 
the  standard  deviations  of  the  night  approaches  more  than  twice  that  of  the  daylight  approaches. 
This  situation  resulted  in  the  necessity  of  attempting  to  make  large  corrections  in  the  final  stages  of 
the  approach,  an  obviously  undesirable  predicament. 

Although  the  problems  with  night  landings  may  be  more  critical  for  the  carrier  situation,  they 
are  of  no  less  concern  to  the  general  aviation  community.  The  specific  behavioral  problems  may 
differ  between  carrier  landings  and  runway  landings  (Lewis  &  Humphries.  1956),  but  one  trend  is 
consistent,  pilots  fly  lower  with  much  greater  variance  in  their  night  approaches.  The  tendency  to 
land  short  on  runways  as  opposed  to  long  on  carrier  decks  has  been  reported  in  the  safety  literature. 
It  is  not  clear  whether  this  is  actually  representative  of  typical  night  approaches  or  whether  it  is  only- 
representative  of  the  unsuccessful  ones. 
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The  pilot's  inability  to  make  accurate  judgements  of  altitude,  sink  rate,  and  rate  of  closure  was 
identified  as  a  major  contributing  factor  in  many  pilot-error  landing  accidents  (Zeller,  1957).  At 
night,  the  difficulty  of  making  these  judgements  is  increased  due  to  the  absence  or  degraded  nature  of 
the  environmental  cues.  This  situation  has  led  to  the  development  of  many  runway  and  cockpit 
landing  aids.  However,  the  last  portion  of  the  approach  is  dependent  on  visual  flight  restructions  and 
pilot  judgement. 

Advances  in  modern  visual  display  systems  have  made  it  possible  to  simulate  a  variety  of 
environmental  conditions  including  night  scenes.  It  is  now  possible  to  practice  night  flying  in  the 
safety  of  a  simulator.  It  is  also  possible  (a)  to  study  problems  involved  in  night  flying,  (b)  to 
determine  the  visual  cues  required  for  effective  simulation,  and  (c)  to  develop  effective  training 
techniques  aimed  at  increasing  transfer  benefits  and  safety. 

Research  efforts  have  begun  in  an  attempt  to  improve  visual  scene  content  and  to  define  the 
training  value  of  various  visual  display  systems.  Kraft,  Anderson,  and  Elworth  (1977)  has 
demonstrated  that  pilots  tend  to  overestimate  their  altitude  when  lights  in  a  computer-generated 
night  scene  have  equal  luminance,  regardless  of  distance.  The  flightpaths  under  these  conditions 
were  significantly  lower  than  when  the  luminance  of  the  lights  was  attenuated.  The  overestimation  of 
altitude  was  greatest  when  no  texturing  cues  were  added  to  the  overrun  and  runway  scenes.  Kraft  et 
al.  (1977)  also  reported  that  runway  position  at  touchdown  improved  with  practice  with  the 
attenuated  luminances  but  that  the  luminance  and  texturing  variables  did  not  result  in  improved  rate 
of  descent  at  touchdown. 

Buckland  (1979)  has  investigated  the  landing  performance  of  experi-  enced  pilots  in  the 
Advanced  Simulator  for  Pilot  Training  (ASPT)  under  various  detail  and  texturing  conditions.  Two 
of  the  conditions  included  night  scenes  similar  to  the  one  used  in  the  present  study.  The  results 
indicate  that  night  scene  touchdowns  were  harder  than  any  of  the  day  scene  conditions.  The  night 
scene  touchdowns  were  also  farther  from  the  threshold  than  were  the  average  day  scene  touchdowns. 
Buckland  also  reported  that  the  touchdown  vertical  velocity  values  were  substantially  larger  for  all 
of  the  scene  conditions  than  for  actual  flight  data.  This  finding  is  consistent  with  the  results  reported 
by  Palmer  and  Cronn  (1976)  in  a  study  of  touchdown  performance  in  a  DC-8  simulator  equipped 
with  a  night  computer  graphics  visual  attachment.  The  authors  also  reported  that  the  vertical 
velocity  values  obtained  in  their  study  were  approximately  10%  to  20%  lower  than  values  obtained 
in  an  earlier  experi-  ment  using  a  TV  model  system  (Bray.  1972). 

The  primary  intent  of  the  previously  cited  research  was  to  study  the  characteristics  of  night  (real 
or  simulated)  flying  and  the  problems  associated  with  night  flying.  More  directly  relevant  for  the 
purposes  of  the  present  study  is  research  dealing  with  transfer  of  training  from  night  scene 
simulation  to  daytime  aircraft  flight.  To  date,  only  two  studies  have  addressed  this  issue:  (a)  the 
Navy  (Brictson  &  Burger,  1976)  conducted  an  evaluation  of  the  A-7E  Night  Carrier  Landing 
Trainer  (NCLT)  which  is  equipped  with  a  night  visual  scene,  and  (b)  the  Air  Force  (Thorpe, 
V  arney.  McFadden.  LeMaster.  &  Short,  1978)  conducted  a  comparative  evaluation  of  three  visual 
system  types  for  possible  application  in  the  KC-I35  program.  In  the  transfer-of-training  evaluation 
of  the  Navy  A-7E  NCLT  conducted  by  Brictson  and  Burger  (1976).  half  of  the  subjects  received  80 
NCLT  approaches  prior  to  actual  flight  training.  Their  performance  was  compared  with  a  control 
group  of  pilots  who  did  not  receive  NCLT  approach  training.  Transfer  evaluations  were  made  both 
during  the  field  carrier  landing  training  and  carrier  qualification  phases  on  both  night  and  day 
approaches.  Measures  of  performance  included  radar  reading,  landing  signal  officer  scores,  success/ 
attrition  rate,  wire  arrestment,  bolter  rate,  and  landing  performance  scores.  The  results  indicated 
that  N CLT  training  resulted  in  significantly  better  performance  on  night  approaches,  particularly  on 
vertical  path  control,  had  more  impart  for  new  pilots  than  for  pilots  with  other  aircraft  back¬ 
grounds.  and  resulted  in  significantly  lower  attrition  rates  (8%  vs  44%  ).  However,  the  NCLT 


training  did  not  improve  performance  on  daytime  approaches,  and  this  is  the  most  important  finding 
for  the  present  study  since  it  shows  the  transfer  of  training  was  specific  to  the  nighttime  environment 
and  did  not  generalize  to  the  daytime  condition, 

Thorpe  et  al.  { 1 078)  conducted  a  transfer  of  training  experiment  comparing  three  visual 
systems:  a  TV  Model  Board,  a  Daylight  Color  CGI,  and  a  Night-Only  Point  Light  Source  CGI.  Three 
groups  of  pilots  recently  graduated  from  the  Undergraduate  Pilot  Training  (l!PT)  program  received 
simulator  training  on  the  visual  traffic  pattern,  approach,  and  landing  using  one  of  the  three  visual 
systems  (using  Boeing  707  aircraft  simulators  rented  from  commercial  flight  training  sources). 
Following  the  simulator  training,  the  pilots  flew  two  evaluation  flights  in  the  KC-135  aircraft  in 
daylight  conditions.  Although  there  were  no  significant  differences  observed  during  the  simultor 
training,  the  results  of  the  evaluation  flights  indicated  that  the  pilots  in  the  Day  Color  CGI  Group 
and  the  Night-Only  CGI  Group  performed  better  on  the  final  approach  glidepath  and  landing 
portions  of  the  task  than  did  the  pilots  trained  with  the  TV  Model  Board  system.  The  latter  group 
exhibited  more  extreme  deviations  from  the  glidepath,  especially  lower  glidepaths  during  the  final 
third  of  the  approach  than  did  the  other  two  groups.  A  follow-up  comparison  of  their  scores  in  the 
CCTS  program  revealed  that  a  higher  percentage  of  pilots  in  the  Day  Color  CGI  Group  received 
"Highly  Qualified"  (90%  )  scores  than  did  those  in  the  Night  (50%  )  or  TV  Groups  (40%  ).  Only 
30%  of  normal  CCTS  students  received  this  grade. 

Thus,  while  there  are  simulator  performance  data  and  subjective  opinions  indicating  that  CGI 
systems  (both  day  and  night)  lack  adequate  cues  for  altitude,  sink  rate,  and  closure  rate  estimation, 
the  only  transfer-of-training  research  data  available  indicate  that  the  CGI  systems  (color  day  and 
point  light  night)  are  at  least  capable  of  supporting  daytime  training,  and  may  be  superior  to  the  TV 
model  board.  The  Navy  study  demonstrated  positive  transfer  from  night  simulator  training  to  night 
carrier  approaches,  but  no  effect  was  observed  for  transfer  to  day  approaches.  Equally  important  for 
the  purposes  of  this  report  is  the  finding  that  no  negative  transfer  was  evident  in  the  performance  of 
the  night  group  on  subsequent  day  performance  in  either  study. 

The  Problem 

The  Air  Training  Command  has  recently  introduced  a  new  ground-based  flight  trainer,  the 
Instrument  Flight  Simulator  (IFS).  into  the  UPT  program.  Although  its  primary  purpose  is  to 
support  instrument  skill  acquisition,  a  secondary  purpose  is  to  support  limited  contact  training 
objectives.  Currently,  half  of  the  IFS  cockpits  are  equipped  with  a  look-ahead  field-of-view  TV 
terrain  model  board  visual  system.  Prior  to  making  any  further  procurements  for  the  remaining 
cockpits,  the  Air  Training  Command  requested  an  estimate  of  the  amount  of  transfer  of  training 
which  could  be  achieved  with  a  dusk/night  limited  field-of-view  visual  system.  The  use  of  such  a 
visual  system  could  result  in  significant  cost  savings  if  there  was  no  loss  in  training  effectiveness.  Of 
particular  interest  was  the  issue  of  negative  transfer  potential  in  the  critical  phases  of  flight,  i.e..  the 
takeoff  and  landing  area.  The  present  research  was  undertaken  to  answer  this  question,  as  well  as  to 
provide  a  comparison  between  the  relative  effectiveness  of  the  ASPT  CGI  day  scene  with  the  night 
scene. 

IL  METHOD 

General  Approach 

A  transfer-of-training  design  was  used  to  assess  the  differential  effectiveness  of  simulator 
training  with  a  day  vs.  a  night  CGI  visual  display.  Twenty-four  novice  student  pilots  were  divided 
into  three  groups:  Day.  Night,  and  Control.  The  Day  and  Night  groups  received  three  training 
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missions  in  the  ASPT  on  Takeoff,  Straight-In  Approaeh  and  Landing,  and  Touch-and-Go.  The 
Control  group  received  standard  syllabus  instruction  (i.e.,  no  ASPT).  Transfer  to  the  aircraft  was 
assessed  by  Instructor  Pilots  (IPs)  on  two  aircraft  sorties  performed  during  daylight  conditions. 


Subjects 

Twenty-four  student  pilots  participated  in  this  study.  Twelve  students  each  were  selected  from 
UPT  Class  78-02  and  78-03.  The  following  descriptive  information  characterizes  the  sample:  (a) 
Source  of  commission  —  11  Reserve  Officer  Training  Commission,  13  Air  Force  Academy 
graduates;  (b)  no  previous  navigator  experience;  (c)  X  age  =22.8  (s.d.  1.12,  range  22—27);  (d) 
previous  simulator  experience,  X  =0.25  hours  (s.d.  0.84,  range  0—3)  and  (e)  previous  aircraft 
experience,  X  =32.7  hours  (s.d.  17.8,  range  16.2  —  40).  None  of  the  students  had  previous  jet  aircraft 
experience. 

Instructor  Pilots 

Four  experienced  T-37  IPs  served  as  simulator  instructors  during  the  pretraining  phase  of  the 
study.  The  flight  instructors  normally  assigned  to  the  students  served  as  data  collectors  during  the 
aircraft  portion  of  the  study;  none  of  the  ASPT  instructor  pilots  participated  during  this  phase. 

All  participating  flightline  instructors  received  familiarization  training  in  the  ASPT  on  data 
collection  procedures  to  be  used  in  the  aircraft.  This  training  consisted  of  a  verbal  briefing  followed 
by  practice  in  data  taking  in  the  ASPT.  Each  IP  viewed  two  performances  of  a  Takeoff,  Full  Stop 
Straight-In  Approach  and  Landing,  and  a  Touch-and-Go.  One  performance  was  representative  of  an 
experienced  T-37  IP  and  the  other  of  a  novice  pilot.  These  performances  were  prerecorded  and 
stored  for  demonstrations;  thus,  each  pilot  viewed  the  same  set  of  maneuvers.  The  instructors  were 
thoroughly  debriefed,  and  definite  errors  were  clarified.  The  IPs  were  also  given  a  written 
information  sheet  to  serve  as  a  refresher  prior  to  inflight  data  collection  (see  Appendix  B).  The 
entire  training  period  was  approximately  1.5  hours  in  length.  The  training  was  given  10  days  prior  to 
the  actual  aircraft  data  missions. 


Equipment 

The  ASPT  research  equipment  provides  a  wide  range  of  capabilities  not  previously  provided  in 
any  one  simulator.  A  complete  description  of  ASPT  is  presented  in  Gum,  Albery,  and  Basinger 
(1975).  An  overview  of  the  aspects  of  the  ASPT  most  relevant  to  the  present  study  is  presented  in 
this  section. 

The  ASPT  is  equipped  with  two  T-37  cockpits.  Each  cockpit  has  a  full  field-of-view  visual 
display  (300°  horizontal  by  150°  vertical),  a  CGI  visual  system,  a  six-degrees-of-freedom  (DOF) 
synergistic  platform  motion  system,  and  a  16-panel  pneumatic  G-seat  on  the  left  seat  (student 
position). 

The  visual  display  is  projected  through  seven  cathode  ray  tubes  (CRTs).  The  capacity  for 
displaying  visual  image  detail  is  fixed  and  shared  between  the  two  cockpits.  A  highly  detailed  scene, 
such  as  an  airport,  requires  90%  to  100%  of  the  display  capacity;  thus,  at  the  most,  only  10%  of  the 
capacity  would  be  available  to  the  other  cockpit.  This  amount  would  result  in  inadequate 
representation  of  a  highly  detailed  scene  but  is  adequate  to  display  a  generalized  view  from  altitude 
such  as  a  horizon  and  surface  texture  pattern.  The  visual  system  uses  an  infinity  optics  display  with 
the  exit  pupil  located  at  the  student's  eye  position.  This  arrangement  results  in  an  optimal  visual 
scene  from  the  student  position,  but  a  distorted  scene  from  the  IP  position.  From  a  normal  position. 
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the  IP  is  unable  to  see  the  visual  display  immediately  in  front  of  the  aircraft.  The  scene  becomes  less 
distorted  as  the  IP  scans  laterally.  By  moving  the  head  position  nearer  to  that  of  the  student’s,  the  IP 
can  increase  the  forward  looking  view  and  reduce  the  distortion. 
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The  platform  motion  system  is  driven  by  six  hydraulic  actuators,  each  with  a  travel  capability  of 
60  inches.  The  platform  motion  system  software  was  designed  to  provide  translational  and  rotational 
acceleration  onset  cues  to  the  student  pilot  position.  The  G-seat  can  also  display  sustained 
accelerating  cues;  however,  the  G-seat  was  not  used  in  this  study  and  will  not  be  discussed.  The 
motion  system  also  includes  a  special  effects  package  which  is  used  to  display  such  cues  as  touchdown 
bump,  runway  rumble,  aircraft  buffet,  speedbrake  extension,  and  gear-down  rumble. 

The  ASPT  has  the  capability  of  real-time  automated  measurement  of  the  pilot’s  performance. 
Measurements  can  be  made  of  pilot  inputs,  system  outputs,  and  scores  can  be  derived  from  these 
measures.  The  measurement  schemes  or  algorithms  for  a  given  maneuver  can  be  preprogrammed. 
An  entire  sortie  content  can  be  preprogrammed  with  automated  performance  measurement  taking 
place  on  predesignated  trials.  A  limited  amount  of  ’.his  information  can  be  displayed  real-time  in  the 
cockpit  via  a  monitor  located  to  the  right  of  the  IP  position  and/or  following  the  mission  in  hard 
copy  form. 

The  ASPT  is  equipped  with  the  capability  of  displaying  a  prerecorded  demonstration  of  a 
maneuver.  The  information  is  stored  on  disc  and  replay  involves  reproduction  of  the  entire 
maneuver  as  originally  recorded,  including  visual  display,  motion  cues,  instrument  readings,  rudder 
and  throttle  movements.  Both  the  students  and  the  IPs  utilized  this  capability  during  the  pretraining 
and  ASPT  phases  of  this  study. 


Visual  Display 

The  data  base  developed  for  the  night  visual  study  consists  of  a  night  airport  traffic  area 
configured  to  closely  approximate  the  M cDonnell-Douglas  Vital  III  display  and  equivalent  day 
airport  traffic  area. 

A  computer-controlled  mask  restricted  the  field  of  view  to  48°  horizontally  and  36°  vertically. 
A  generalized  airport  scene  was  created  with  one  model  depicting  the  night  scene  and  another  model 
depicting  the  day  scene.  For  the  night  scene,  the  environment  included  a  6.000-foot  lighted  runway 
with  approach  lighting  and  runway  markings.  The  runway  centerline  was  visible  for  1.000  to  2.000 
feet  in  front  of  the  aircraft  on  the  runway,  with  touchdown  zone  stripes  at  the  approach  end  of  the 
runway.  Inset  runway  lights  were  also  provided  to  simulate  the  runway  lighting  of  the  M cDonnell- 
Douglas  \  ital  111  system.  There  were  base  lights  randomly  positioned  out  to  10.000  feet  on  both  sides 
of  the  runway  and  from  10.000  feet  out  from  the  departure  end  of  the  runwav.  Horizon  lights  were 
also  randomly  positioned  at  a  further  distance,  providing  the  appearance  of  convergence  as  the 
aircraft  descended  to  lower  visual  approach  angles.  Horizon  glow  was  added  providing  a  dusk/night 
scene.  Other  visual  details  included  a  cube  tower  with  rotating  beacon  on  top.  offset  5.000  feet  to  the 
left  of  the  runway.  There  were  two  radio  towers  offset  3.000  feet  ti.  the  right  of  the  runway  and  two 
light  towers  on  the  right  side  of  the  runway  offset  2.000  and  1.000  feet  from  the  end  of  the  runway. 

The  day  scene  consisted  of  a  fully  marked  (>.000  foot  runwav  with  runway  edge  lights  and 
sequenced  flashing  approaching  the  runway.  Low  level  velocity  and  altitude  rue  enhancement  was 
achieved  through  the  use  of  numerous  three-dimensional  features  in  this  area.  On  final  approach, 
there  was  a  drive-in  movie  theatre  and  an  agricultural  area  consisting  of  a  church,  farm  buildings, 
pickup  truck,  wagon,  and  tractor  with  plow.  The  control  tower  and  other  buildings  were  adjacent  to 
the  runway,  with  a  factory  in  the  background,  and  at  the  threshold,  a  T-37  aircraft  was  waiting  to  take 
the  active  runway. 
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Student  Training 

Following  selection  for  study  participation,  the  subjects  were  randomly  assigned  to  one  of  three 
groups:  Day.  Night,  or  Control.  Subjects  in  the  Control  group  did  not  receive  any  ASPT  pretraining 
prior  to  entry  into  the  T-37  flying  phase.  Students  in  the  two  experimental  groups  received  three 
ASPT  sorties  in  which  instruction  was  given  on  Takeoffs.  Full  Stop  Straight-In  Approaches,  and 
Straight-In  Approaches  to  a  Touch-and-Go  Takeoff. 

The  visual  field  of  view  was  limited  to  48°  horizontal  by  36°  vertical  by  a  computer-generated 
mask.  This  was  essentially  a  look-ahead  view  designed  to  represent  the  field-of-view  available  on  the 
IFS  in  use  by  the  Air  Training  Command.  The  ASPT  syllabus  consisted  of  three  sorties,  each 
approximately  one  hour  in  length.  The  first  and  second  sorties  were  separated  by  a  48-hour  interval, 
with  the  second  and  third  sorties  occurring  in  daily  succession.  The  content  of  each  sortie  was 
specified  in  terms  of  the  number  of  repetitions  per  task  and  task  order.  The  ASPT  syllabus  is 
presented  in  Appendix  A.  A  total  of  13  Takeoffs,  13  Straight-In  Approach/Landings.  and  9  Straight- 
In  Approaeh/Toueh-and-Go  maneuvers  were  performed.  Following  completion  of  the  three  ASPT 
missions  and  standard  syllabus  requirements,  the  students  began  flightline  training.  The  desired  time 
between  the  last  ASPT  mission  and  the  first  aircraft  flight  was  1  dav.  however,  in  some  cases  the 
interval  was  as  long  as  3  days  because  ASPT  training  was  completed  on  a  Friday. 

Tra  ns  fe  r  Eva  iua  dons 

The  first  T-37  aircraft  flight  was  conducted  as  specified  in  the  standard  L  PT  syllabus  with  the 
proviso  that  a  demonstration  of  a  Takeoff.  Straight-ln  Approach,  and  Full  Stop  Landing  and  a 
Straight-In  Approach  to  a  Touch-and-Go  Takeoff  was  given  by  the  IP  with  no  "hands  on"  practice 
by  the  student. 

The  second  and  fifth  T-37  flights  served  as  the  data  collection  missions  for  all  three  groups.  The 
IPs  were  asked  to  design  each  mission  to  include  at  least  one  hands-on  repetition  by  the  student  of 
the  Initial  Takeoff.  Straight-ln  Approach  to  a  Touch-and-Go.  and  a  Straight-In  Approach  to  a  Full 
Stop.  Performance  evaluation  data  were  to  he  recorded  on  each  task  by  the  IP  as  soon  as  possible 
following  task  completion.  In  the  event  that  more  than  one  repetition  of  any  of  the  tasks  was 
performed,  the  IP  was  asked  to  collect  the  desired  data  on  these  additional  repetitions  as  well.  The  IP 
was  asked  to  refrain  from  instructing  during  the  performance  of  each  task:  however,  in  no  case  was 
the  safety  of  the  flight  to  he  compromised.  In  the  event  that  the  IP  had  to  take  control  of  the  aircraft, 
only  the  portions  of  the  flight  flown  by  the  student  were  to  be  recorded. 

Dependent  Measures 

Performance  measures  of  two  types  were  collected  on  each  trial  designated  as  a  measurement 
trial.  (No  performance  measures  were  collected  on  the  remaining  trials.)  Root-mean-square  error 
measures  were  collected  on  each  task  parameter  which  had  a  criterion-referenced  objective 

I n  addition  to  the  objective  automated  measures,  subjective  performance  ratings  were  obtained 
from  the  IP.  The  basic  rating  scale  was  the  same  used  in  the  norma)  ATC.  training  program.  This 
scale  specifies  standards  for  each  grade  of  T  (T  n satisfactory).  F  (Fair),  G  (Good),  and  E  (Excellent). 
For  the  purposes  of  this  study,  this  scale  was  subdivided  into  an  Absolute  scale,  corresponding  to 
published  performance  standards,  and  a  Relative  scale.  The  IP  was  instructed  to  use  the  Relative 
scale  by  assigning  a  grade  based  upon  a  comparison  of  the  student's  performance  to  that  of  other 
students  at  the  same  point  in  training.  The  standard  of  comparison  for  the  Relative  scale  was  totally 
subjective,  based  on  the  IP's  accumulated  teaching  experience.  This  dual  standard  scale  was  used  in 


an  attempt  to  reduce  the  variance  typically  encountered  in  rating  scale  judgements.  Previous 
research  suggested  the  use  of  the  dual  scale  would  provide  increased  measurement  sensitivity  as  well 
as  reducing  inter-  and  intra-rater  variability. 

The  original  study  protocol  included  use  of  the  task  observation  data  to  be  collected  in  both  the 
ASPT  and  the  T-37.  However,  during  IP  pretraining  sessions,  it  was  shown  to  be  extremely  difficult 
to  make  accurate  observations  of  runway  alignment  from  the  right  (IP  position)  of  the  ASPT  due  to 
the  distortion  of  the  visual  display  from  that  position.  Therefore,  these  data  were  collected  only  on 
the  two  T-37  evaluation  flights.  The  data  card  formats  are  presented  in  Appendix  C. 

Data  Analysis 

The  automated  data  collected  in  the  ASPT  was  analyzed  using  a  multivariate  analysis  of 
variance  technique  (M  ANOVA)  which  provided  an  overall  test  of  significance  (Wilks-Lambda)  and 
univariate  stepdown  F  tests  for  each  of  the  individual  parameters.  The  RMS  error  scores  were 
logarithmically  transformed  prior  to  analysis.  Due  to  several  bad  data  points  and  the  unavailability 
of  appropriate  statistical  programs,  each  measured  repetition  of  each  task  was  subjected  to  a  separate 
M  ANOVA.  The  confidence  limit  established  for  the  Wilks-Lambda  was  p  <.10. 

Relative  and  Absolute  scale  ratings  of  task  proficiency  given  in  the  U,  F,  G,  and  E  format  were 
transformed  into  integer  values  1,  2,  3,  and  4,  respectively.  For  ratings  collected  in  the  ASPT,  data 
from  each  scale  type  were  analyzed  separately  using  a  mixed  design  analysis  of  variance  (ANOVA). 
From  the  ratings  collected  in  the  T-37,  only  Relative  scale  data  were  subjected  to  statistical  analysis 
since  visual  inspection  of  the  Absolute  scale  data  revealed  no  mean  differences  with  almost  no 
variability.  Due  to  incomplete  data,  a  simple  ANOVA  was  performed  for  each  repetition  of  each  task. 
The  confidence  limit  was  set  at  .05. 

The  original  intent  was  to  use  a  chi  square  analysis  for  each  item  of  the  categorical  data  collected 
on  the  evaluation  flights.  However,  there  were  too  many  cells  containing  expected  frequencies  of 
zero.  Therefore,  the  entry  for  each  item  from  each  task  on  each  flight  was  converted  to  either 
"correct"'  or  "incorrect."  Thus,  either  a  “high"  or  “low"  data  entry  was  treated  as  “incorrect."  The 
frequencies  of  “correct”  and  “incorrect"  items  were  summed  across  all  the  items  forming  the  basis  of 
a  two  (Correct  vs.  Incorrect)  by  three  (Day  vs.  Night  vs.  Control)  contingency  table.  A  separate 
contingency  table  and  associated  analysis  was  computed  for  each  task  on  each  flight.  The  chi  square 
statistic  tests  the  likelihood  that  the  observed  distributions  were  due  to  chance.  The  coefficient  of 
contingency,  C.  was  also  computed  for  each  table.  This  measure  gives  an  indication  of  the  degree  of 
association  between  the  groups  and  their  performance. 


ffl.  RESULTS 

ASPT  Data 

Three  questions  are  of  interest  when  examining  the  outcomes  o  f  the  ASPT  training,  (a)  Did  the 
students'  performance  improve  during  the  ASPT  training?  (b)  Were  there  any  performance 
differences  as  a  function  of  the  day  versus  night  scenes  used  during  training?  (e)  Did  the  students' 
performance  improve  differentially  as  a  function  of  the  day  versus  night  scenes?  Two  types  of 
dependent  measures  are  available  to  address  these  questions:  I P  performance  ratings  and  automated 
system  state  measures  of  task  performance. 
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Table  l  presents  mean  values  of  the  performance  ratings  by  task,  group,  scale  type,  and 
repetition  number.  These  data  were  analyzed  using  a  mixed  design  analysis  of  variance  (ANOVA) 
with  Day  versus  Night  groups  as  the  between-group  factor  and  trials  as  the  repeated  measures  factor. 
The  resulting  F  statistics  and  associated  probability  levels  are  presented  in  Table  2. 

Table  1.  Instructor  Pilots' Mean  Proficiency  Ratings:  ASPTTraining 


Trial  1 

Trial  2 

Trial  3 

Task  Condition 

Re  la  live 

Absolute 

Re  la  tive 

Absolute 

Relative 

Absolute 

Day 

2.25 

1.25 

2.875 

2.125 

3.125 

2.50 

Takeoff 

Night 

2.75 

1.50 

2.875 

2.125 

3.25 

2.75 

Full  Stop 

Day 

1.875 

1.375 

3.375 

2.50 

Straight-In 

Approach 

Night 

2.50 

1.875 

3.00 

2.50 

Approach 

Portion  of 

Day 

2.875 

2.50 

3.50 

2.875 

Touch-and-Go 

N  ight 

3.125 

2.50 

3.125 

2.625 

Takeoff 

Day 

2.75 

2.125 

3.25 

3.0 

Portion  of 
Touch-and-Go 

Night 

3.125 

2.25 

3.125 

2.75 

Table  2.  ASPT  Training :  IP  Ratings  -  FStatisties 


Task 

Scale 

Group  (D  vs  N) 

Trials 

Group  x  Trials 

Takeoff 

R 

<i 

4.550* 

<i 

A 

<i 

18.034* 

<i 

Full  Stop  Str-In 

R 

<i 

11.200* 

2.80 

A 

<i 

13.451* 

1.098 

Str-In  App 

R 

<i 

1 .000 

1.000 

A 

<i 

<1 

<1 

Touch-and-Go 

R 

<i 

2.333 

2.333 

A 

<i 

11.930* 

<1 

*p  <.01. 
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The  data  collected  by  the  ASPT  automated  performance  measurement  (APM)  system  were 
analyzed  using  a  M  ANOV  A.  The  means  and  standard  deviations  of  the  APM  data  and  the  results  of 
the  associated  statistical  analyses  are  presented  by  task  in  Tables  3  (T akeoff).  4  (Full  Stop  Straight-1  n 
Approach),  and  5  (Straight-In  Approach/Touch-and-Do).  Due  to  occasional  equipment  failure,  some 
of  the  data  were  lost.  For  this  reason,  each  repetition  of  each  task  was  analyzed  separately.  Rather 
than  estimating  mission  data,  the  data  were  deleted  for  a  subject  selected  at  random  from  the 
opposite  group.  In  these  cases,  n  =  7  instead  of  8. 


Table  3.  Descriptive  and  Inferential  Statistics 
ASPT/Automated  Measures :  Takeoff 


Rotation 

Liftoff 

Heading 

Attitude 

Altitude 

Speed 

Speed 

Error 

Error 

Error 

Trial  1 

Day  X 

77.62 

87.74 

.968 

.860 

5.56 

SD 

6.33 

2.38 

.756 

.292 

.56 

Night  X 

75.14 

84.74 

.574 

.878 

5.59 

SD 

6.71 

4.16 

.828 

.504 

.39 

F(l.  12) 

<1 

2.728 

<1 

<1 

<1 

n  =7.  Lambda 

-Wilks  F(5  8)  <1 

Trial  2  __ 

Dav  X 

78.06 

86.65 

.660 

.842 

5.38 

SD 

6.29 

5.36 

.521 

.471 

73 

Night  X 

77.29 

86.91 

.296 

.620 

5.1 7 

SD 

9.23 

6.92 

.504 

.266 

.49 

F(l.  14) 

<1 

<1 

2.016 

1 .350 

<1 

n  =8.  Lambda 

-Wilks  F(f>  ,0)  <1 

Trial  ,3 

Dav  X 

78.44 

88.03 

.354 

.577 

5.14 

SD 

7.32 

4.35 

.598 

.335 

.33 

Night  X 

72.26 

84.50 

.411 

.278 

5.20 

SD 

5.36 

4.31 

.480 

1.24 

.57 

F(l,  14) 

3.711* 

2.670 

<1 

<1 

<i 

n  =8.  Lambda 

-  Wilks  f(5  10)  <1 

*|>  <.lo. 


Table  4.  Descriptive  and  Inferential  Statistics  ASPT/Automated  Measure 
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The  results  of  the  performance  rating  analyses  provide  information  with  respect  to  all  three  of 
the  questions  of  interest,  whereas  the  analyses  of  the  APM  data  are  only  relevant  to  the  Day  versus 
Night  comparisons.  The  results  of  the  ASPT  training  phase  may  be  summarized  as  follows,  (a) 
Student  performance  improved  significantly  on  the  Takeoff,  Full  Stop  Straight-ln,  and  the  Takeoff 
portion  of  the  Toueh-and-Go.  (b)  There  were  no  differences  between  the  Day  and  Night  groups  as 
assessed  by  instructor  pilot  performance  ratings.  Analyses  of  the  APM  data  indicated  initial 
superiority  of  the  N  ight  group  on  the  single  parameter  of  control  on  the  glidepath  angle  of  descent  on 
the  Full-Stop  Straight-In  Approach.  However,  the  Day  group  was  significantly  better  than  the  Night 
group  on  that  parameter  and  the  task  as  a  whole  by  the  next  measurement  point.  There  were  no 
reliable  performance  differences  on  the  Takeoff  or  Touch-and-Go.  (c)  There  was  no  tendency  for 
performance  to  be  differentially  influenced  by  the  differences  in  day  and  night  scenes. 

T-37  Transfer  Evaluation 

There  are  two  questions  of  primary  interest  concerning  student  performance  in  the  aircraft,  (a) 
Vi  ere  there  any  reliable  differences  between  the  performance  of  the  Day  and  Night  group?  (b)  Did 
either  or  both  of  the  ASPT  trained  groups  perform  better  than  the  Control  group.  Questions 
concerning  negative  transfer  indications  for  the  Night  group  and  learning  differences  between  the 
three  groups  are  also  of  interest.  Deviations  in  the  study  protocol  occurred  which  affected  the  ability 
to  address  these  questions  as  originally  planned.  These  items  will  be  discussed  briefly  before 
proceeding  with  a  presentation  of  the  data. 

As  originally  designed,  the  transfer  evaluations  were  to  consist  of  at  least  one  repetition  of  each 
task  on  each  of  the  two  evaluation  flights.  However,  for  various  reasons,  these  goals  were  not 
consistently  accomplished.  The  biggest  problem  occurred  on  the  Approach  portion  of  the  Full  Stop 
or  Touch-and-Go  tasks.  Apparently,  the  students  were  given  the  opportunity  to  fly  one  or  the  other, 
but  not  both  maneuvers.  The  IPs  were  more  willing  to  allow  the  students  to  fly  the  takeoff  portion  of 
the  Touch-and-Go.  As  a  consequence,  the  data  from  the  Approach  segments  of  the  Full  Stop  and 
Touch-and-Go  were  collapsed  and  treated  as  the  same  task.  Another  consequence  of  the  incomplete 
data  collection  was  the  inability  to  use  a  mixed  design  ANOV  A  on  the  rating  data.  Separate  A  NOV  As 
of  each  repetition  were  used  instead. 

Two  types  of  information  were  collected  by  IPs:  (a)  relative  and  Absolute  performance  ratings 
as  used  in  the  ASPT  phase,  and  (b)  criterion-referenced  observations  of  aircraft  control.  (In  some 
instances,  the  I P  marked  more  than  one  alternative  for  a  given  item.  e.g..  circled  -50.  0.  and  +  50  feet 
on  altitude  control.  When  this  occurred,  each  circled  option  was  treated  as  an  integer  for  each 
category  in  the  contingency  tables.)  Tables  6.  7.  and  8  present  the  2  by  3  contingency  tables  and 
associated  statistics.  Table  0  presents  the  descriptive  and  inferential  statistics  for  performance 
ratings. 

I)uy  vs.  ^  ight.  The  results  of  the  a  priori  “t  "  tests  comparing  the  performance  ratings 
( R dative  scale  only)  are  the  only  direct  source  of  information  regarding  the  differences  between  the 
Day  and  Night  groups.  The  results  of  these  analyses  indicate  that  there  were  no  significant 
differences  between  the  groups  on  any  of  the  tasks  on  either  evaluation  flight. 

I  raining  I,  ffvr  tiv  eness.  Comparison  of  the  performance  of  the  Control  group  which  did  not 
receive  ASPT  training  with  the  performance  of  the  ASPT-trained  students  provides  the  indication 
of  training  effectiveness.  The  results  of  the  a  priori  “l  tests  of  the  Day  and  Night  groups  combined 
v  ersus  the  ( .ontrol  group,  the  “F  tests,  and  to  an  extent,  the  chi  square  analvses  provide  information 
regarding  this  issue.  However,  the  rating  data  obtained  on  the  Absolute  scale  is  equally  valuable  in 


Table  6.  Task  Item  Analysis :  Takeoff 


CoiTCCl 

Incoirect 

a. 

First  observation 

Control 

10 

38 

Night 

16 

31 

Day 

15 

32 

df 

=2,  x2  =2  33  (p  <.50),  C  =.13 

b. 

Second  observation 

Control 

14 

34 

Night 

27 

21 

Day 

28 

22 

df 

=2,  x2  =8.75  (p  <.02),  C  =.24 

Table  7.  Task  Item  Analysis: 

Straight-In  Approach 

Correct 

Incorrect 

a. 

First  observation 

Control 

17 

55 

Night 

22 

52 

Day 

28 

44 

df 

=  2,  x2  =4.00  (p  <10),C  =.13 

b. 

Second  observation 

Control 

27 

51 

Night 

35 

43 

Day 

26 

33 

df 

=  2.  x2  =8.75  (p  <.02),  C  =.10 

J  7 


Table  8.  Task  hem  Analysis :  Touch-and-Go/ 
Takeoff  Segment 


Correct 

Incorrect 

a. 

First  observation 

Control 

8 

26 

N  ight 

14 

34 

Day 

14 

21 

df 

=2.  x2  =2.29  (p  <50),  C  =.14 

b. 

Second  observation 

Control 

14 

42 

Night 

26 

23 

Day 

11 

31 

df 

=2,  x2  =10.96  (p  <.01),  C  =.24 

Table  9.  Instructor  Pilot  Mean  Ratings  and  Analysis 
T-37 


Task 

Trial 

Scale 

Control 

Night 

Day 

F 

t(D  vs  N) 

T(D/N  vs  C) 

l 

K 

1.625  (8)* 

2.125  (8) 

2.714  (7) 

3.638** 

1.270 

-2.2009** 

Takeoff 

A 

1.0 

1.0 

1.25 

NC** 

NC 

NC 

2 

R 

2.00  (8) 

2.625  (8) 

2.857  (7) 

2.041 

<1 

-1.086** 

A 

1.50 

2.12 

1.75 

NC 

NC 

NC 

i 

R 

1.625  (8) 

1.750  (8) 

2.125  (8) 

<1 

<1 

<1 

Straight-In 

A 

1.0 

1.0 

1.0 

NC 

NC 

NC 

A pproach 

2 

R 

2.125  (8) 

2.750  (8) 

2.420  (7) 

<1 

<1 

-1.157** 

\ 

1 .37 

2.00 

1.57 

NC 

NC 

NC 

Takeoff 

1 

R 

l.h  (5) 

1.857  (7) 

2.20  (5) 

<1 

<1 

<1 

Portion 

A 

1.0 

1.0 

1.0 

NC 

NC 

NC 

of  Touch* 

2 

R 

1.875  (8) 

1 .857  (7) 

2.167  (6) 

<1 

<1 

<1 

and-Co 

A 

1.120 

1.0 

1.33 

NC 

NC 

NC 

•Sample  size. 
**p  <05. 

•••Not  computed. 
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this  regard.  As  far  as  the  Absolute  rating  is  concerned,  there  was  virtually  no  positive  (or  negative) 
transfer  of  training  effect.  The  results  of  the  “t”  and  “F”  tests  on  the  R  elative  rating  data  reveal  that 
positive  transfer  occurred  only  on  the  Takeoff  task.  From  the  chi  square  analyses,  the  only  instance 
which  revealed  a  non-chance  distribution  was  the  Takeoff  portion  of  the  Touch-and-Go  task. 
Although  more  difficult  to  interpret,  it  would  appear  that  the  performance  of  the  Night  group  was 
superior  to  that  of  the  Control  group.  This  difference  is  not  consistent  with  the  ordinal  rankings  of 
the  groups  on  the  rating  data. 

W  ith  respect  to  negative  transfer,  there  were  no  indications  from  the  data  or  from  the  postflight 
interviews  with  the  IPs  that  any  potentially  hazardous  flying  skills  resulted  from  the  ASPT  training 
(Day  or  Night  conditions). 

Learning.  Due  to  the  incomplete  data  returns,  the  statistical  techniques  used  do  not  provide  a 
direct  test  of  the  magnitude  of  skill  acquisition  between  the  two  evaluation  flights.  It  is  evident  from 
a  visual  inspection  of  the  data  that  all  three  groups  improved.  The  more  interesting  experimental 
question  concerns  the  differential  learning  rates  that  may  be  associated  with  the  ASPT  training, per 
se,  or  the  type  of  visual  display.  It  appears  that  the  Day  and  Control  groups  improved  approximately 
equivalent  amounts  from  the  first  to  second  evaluation  flight,  with  the  Night  group  showing  slightly 
larger  gains  for  the  Takeoff  and  Straight-In  Approach  tasks  on  the  Absolute  scale. 


IV.  DISC  CSS  ION 


The  study  was  designed  to  provide  preliminary  information  regarding  the  transfer  of  training  of 
a  restricted  field-of-view  night  visual  scene  for  application  to  the  IFS  used  for  UPT.  In  addition,  a 
comparison  was  made  between  the  night  scene  and  the  day  scene  available  on  the  ASPT.  Both  visual 
scenes  were  produced  by  CGI  system. 

In  order  to  determine  the  transfer  value  of  the  night  scene  or  to  assess  the  differential 
effectiveness  of  the  day  versus  night  scenes,  transfer  of  training  must  first  be  demonstrated.  This  was 
accomplished  by  comparing  the  performance  in  the  aircraft  of  students  who  were  trained  in  the 
ASPT  (Day  or  Night  condition)  with  the  performance  of  students  who  did  not  receive  ASPT 
training,  i.e..  the  Control  group.  The  first  issue,  then,  to  be  addressed  concerns  the  evidence  that  any 
transfer  of  training  actually  occurred. 

The  data  on  the  performance  ratings  provided  on  the  Absolute  scale  do  not  show  any  difference 
in  performance  of  the  three  groups.  The  initial  performance  of  all  three  groups  on  all  three  tasks  was 
judged  to  be  unsatisfactory.  Although  performance  ratings  increased  somewhat  by  the  fifth  flight, 
there  was  very  little  difference  between  the  three  groups.  Thus,  according  to  this  data  source,  one 
would  have  to  conclude  that  no  transfer  of  training  occurred.  The  data  obtained  on  the  Relative  scale 
and  the  task  observation  information  reflect  greater  sensitivity  to  group  differences  and  training 
effects.  The  results  of  analyses  on  these  data  indicate  that  the  two  groups  trained  in  the  ASPT 
performed  the  Takeoff  task  reliably  better  than  did  the  Control  group.  Their  performance  on  the 
other  tasks  tended  to  be  rated  higher  than  the  Control  group  but  these  differences  were  not 
statistically  significant.  The  most  reasonable  conclusion  regarding  the  extent  of  transfer 
demonstrated  in  this  study  would  be  that  a  small  positive  transfer  effect  was  demonstrated  for  the 
Takeoff  task. 

The  relatively  small  magnitude  of  demonstrated  transfer  is  particularly  surprising  in  light  of  the 
fact  that  significant  improvement  in  performance  was  demonstrated  by  both  experimental  groups 
during  the  ASPT  training  phase.  By  the  end  of  the  ASPT  training,  student  performance  was  rated 
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high  in  the  Fair  range  on  the  Absolute  scale  and  in  the  Good  range  on  the  K  elative  scale.  Thus,  while 
student  performance  and  skill  acquisition  seemed  satisfactory  during  the  ASPT  training,  the 
performance  levels  were  not  sustained  in  transition  to  aircraft  flight.  Clearly,  the  transfer  was  not  on 
a  one-to-one  basis  as  many  training  program  designs  expect  prior  to  the  introduction  of  a  modern 
simulation  device  into  a  training  program. 

If  transfer  is  less  than  the  one-for-one  tradeoff  or  if  the  simulator  training  does  not  elevate 
performance  in  the  aircraft  by  at  least  a  full  grade  on  the  operational  grading  scale,  the  device  is  often 
considered  worthless  by  the  operational  training  community.  However,  often  times  such 
disappointing  findings  are  the  result  of  inadequate  performance  assessment  techniques  and  of 
underestimating  the  importance  of  training  technology.  The  operational  grading  scales  are  generally 
designed  to  detect  large  changes  in  proficiency  and  are,  by  design,  relatively  insensitive  to  smaller 
changes  in  performance.  Thus,  while  simulator  training  may  not  tesult  in  a  full  grade  change  in 
performance  (as  from  Unsatisfactory  to  Fair),  the  training  may  elevate  performance  within  a 
category  (from  the  bottom  to  the  top  of  a  grade  category).  These  differences  may  have  significant 
pragmatic  implications  for  a  training  program,  especially  if  similar  increments  occurred 
continuously  throughout  each  block  or  phase  of  a  syllabus.  However,  by  many  current  operational 
measurements  standards,  these  effects  would  not  be  detected.  This  kind  of  problem  is  particularly 
critical  to  design  and  procurement  decisions  based  on  the  results  of  the  Operational  Test  and 
E  valuation  process.  The  results  of  the  present  study  reflect  a  small  transfer  effect  which  in  itself  may 
be  trivial  but  could  have  a  significant  cumulative  effect  when  spread  over  the  entire  basic  contact 
phase  of  training.  The  results  of  this  study  also  demonstrate  the  difference  in  sensitivity  of  varying 
assessment  techniques. 

W  ith  respect  to  the  effectiveness  of  the  night  scene  for  application  to  daytime  flight  training,  the 
results  show  that  the  Night  group  consistently  performed  better  than  the  Control  group  although  the 
differences  were  small  for  the  Straight-In  and  Touch-and-Go.  There  were  no  indications  of 
hazardous  elements  of  either  the  glidepath  or  flare  segments  of  the  approach.  However,  given  the 
relatively  low  level  of  transfer  obtained  in  the  present  study,  it  is  difficult  to  draw  firm  conclusions 
regarding  the  training  potential  of  the  night  scene.  Improvements  in  simulator  training  strategies  can 
reasonably  be  expected  to  enchace  the  magnitude  of  transfer,  but  it  is  not  clear  to  what  extent.  It  is 
difficult,  if  not  impossible,  to  make  an  evaluation  of  any  training  device  independent  of  the  training 
methodology  used  in  the  evaluation. 

A  comparison  between  the  night  scene  and  the  model  board  scene  would  be  particularly  useful 
considering  the  results  of  the  Navy  NCL,T  study  (Brictson  &  Burger.  1 976)  and  that  of  the  A  ir  Force 
KC-135  study  (Thorpe  et  al..  1978).  The  Navy  study  demonstrated  a  large  positive  transfer  effect 
from  the  simulator  to  night  flight  conditions  but  no  transfer  effect  to  daytime  conditions  i.e..  transfer 
specific  to  night  flight.  The  Air  Force  study  offered  evidence  that  the  model  board  system  was 
inferior  to  day  and  night  CGI  systems  on  the  final  approach  maneuver.  Since  the  Air  Force  study  did 
not  employ  a  control  group,  it  is  not  possible  to  assess  the  overall  magnitude  of  transfer.  However, 
comparisons  were  made  late  in  the  operational  training  program  with  a  pseudo-control  indicating 
that  all  the  groups  which  received  the  simulator  pretraining  performed  better  than  pilots  who  had 
not  participated  in  the  study.  Additionally,  it  was  found  that  the  day-color  CGI  group  had 
maintained  a  superior  level  of  performance  when  compared  to  the  night  and  model  board  groups. 

Considering  the  data  available  to  date,  it  is  unclear  how  effective  a  simulated  night  visual  scene 
is  for  transfer  to  day  flight.  The  present  study  demonstrates  a  small  level  of  transfer:  the  KC-135 
shows  a  larger  level  of  transfer:  the  Navy  A-7  study  did  not  demonstrate  any  positive  daytime  effect, 
but  there  was  a  significant  nighttime  effect.  Equally  as  pertinent  for  Air  Training  Command 
concerns  is  the  fact  that  the  model  board  system,  the  type  currently  in  use  on  the  IFS.  was  not  as 
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effective  as  the  night  scene  in  the  KC-135  study.  Therefore,  the  most  logical  step  for  Air  Training 
Command  would  be  to  conduct  a  direct  model  board/night  scene  comparison  within  the  operational 
training  context. 


V.  CONCLUSIONS/RECOMMENDATIONS 


1.  A  small  positive  iransfer-of-training  effect  was  observed  for  both  the  Day  and  Night 
experimental  groups. 

2.  There  were  no  reliable  differences  between  the  Day  and  Night  groups  in  terms  of  their 
performance  in  the  aircraft. 

3.  There  was  no  evidence  of  negative  transfer  associated  with  the  performance  of  the  night 
group. 

4.  In  order  to  maximize  transfer-of-training  benefits,  implementation  of  a  night  visual  scene 
should  be  associated  with  more  extensive  simulator  training  than  was  provided  in  the  present  study. 
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APPENDIX  A:  ASPT  SCENARIO  CONTENT 


MISSION  1 

TAKEOFF 

DEMONSTRATION 

TAKEOFF 

3 

TAKEOFF 

APM 

STRAIGHT-IN 

FS 

DEMONSTRATION 

STRAIGHT-IN 

FS 

3 

STRAIGHT-IN 

FS 

APM 

TAKEOFF 

2 

TAKEOFF 

APM 

MISSION  2 

TAKEOFF 

2 

STR AIGHT-IN  FS 

2 

STRAIGHT-IN  TG 

DEMONSTRATION 

STR  AIGHT-IN  TG 

3 

STRAIGHT-IN  TG 

APM 

STRAIGHT-IN  FS 

2 

MISSKIN  3 

TAKEOFF 

3 

TAKEOFF 

APM 

STRAIGHT-IN  FS 

4 

STRAIGHT-IN  FS 

APM 

STRAIGHT-IN  TG 

4 

STRAIGHT-IN  TG 

APM 

APPENDIX  B:  IP  INFORMATION  GUIDE  NIGHT  VISUAL  SCENE  EVALUATION 


The  objective  of  this  study  is  to  evaluate  the  training  value  of  a  night  visual  simulator  scene  for 
daytime  aircraft  training.  As  the  instructor  pilot,  you  are  the  primary  evaluator.  The  purpose  of  this 
guide  is  to  introduce  you  to  the  study  objectives,  make  you  aware  of  the  observations  that  must  be 
made  by  you  to  effectively  evaluate  the  student  and  to  help  you  understand  the  use  of  the  in-cockpit 
grade  cards. 

W e  are  asking  that  you  make  rather  detailed  observations  on  the  student’s  performance  on  three 
maneuvers:  takeoff,  straight-in  approach  and  landing,  and  the  straight-in’s  in  combination  with  the 
touch-and-go.  These  observations  will  be  collected  by  you  on  the  second  (B 2 1 0 1 )  and  fifth  (B2202) 
sorties.  It  is  essential  that  the  student  not  have  any  "hands  on”  control  at  these  tasks  until  the  second 
(B2101)  sortie.  In  addition  to  collecting  the  maneuver  specific  data,  you  will  be  interviewed 
following  the  student's  fifth  sortie  on  your  subjective  comments  concerning  the  student’s 
performance.  These  interviews  are  for  research  purposes  only  and  will  be  kept  strictly  confidential. 


Limits  of  the  Training  in  ASPT: 

1.  No  X-wind  training 

2.  Little  procedural  training 

3.  Takeoff  continued  to  approximately  1900’  MSL 

4.  All  landings  trained  to  the  center  of  the  runway 

5.  Straight-in  training  initiated  at  approximately  five  miles  out. 

NOTE:  Students  who  have  been  trained  in  ASPT  will  have  had  extensive  straight-in,  takeoff,  and 
touch-and-go  training,  so  please  allow  those  particular  students  to  fly  with  as  little  assistance  as 
possible,  but  of  course  not  to  com  pro  m ise  flying  safety ! 

I  have  included  an  enlargement  of  the  takeoff,  straight-in  approach  and  landing,  and  touch-and- 
go  grade  cards  that  you  will  use  in  the  aircraft.  W e  would  like  the  data  recorded  on  these  cards  as  soon 
as  possible  after  the  student  performs  the  maneuver:  however,  do  not  sacrifice  clearing  or  aircraft 
control  of  any  kind  to  complete  them.  The  cards  are  generally  self-explanatory,  but  some  questions 
have  come  up  regarding  unusual  circumstances.  For  example,  during  the  T/O  groundtrack  control, 
you  should  mark  left.  on.  or  right.  If  the  student  does  not  over  or  under  control,  don’t  mark  the 
corresponding  blocks.  On  the  other  hand,  if  the  student  does  over  or  under  control,  mark  the 
appropriate  box  and  indicate  whether  left.  on.  or  right  best  represents  the  performance.  During  the 
landing  from  the  straight-in.  make  sure  you  indicate  whether  the  flare  was  smooth  or  abrupt  in 
addition  to  whether  the  student  flared  high.  on.  or  late.  Also,  be  sure  to  indicate  whether  the  straight- 
in  was  a  toueh-and-go  or  full  stop.  Use  your  best  judgement  when  filling  out  the  cards  and  make  sure 
that  each  item  has  been  marked.  Any  item  not  graded  will  make  it  very  difficult  for  us  to  evaluate 
vour  student’s  performance.  If  you  have  to  take  aircraft  control  during  any  part  of  the  maneuver. 
please  include  it  in  your  comments  at  the  bottom  of  the  card.  Questions  have  come  up  about  the 
overall  score.  I’m  sure  you  all  understand  the  difference,  but  to  reinforce  youT  understanding,  we 
want  the  absolute  grade  to  reflect  the  student's  performance  against  the  perfect  maneuver.  Relative 
score  should  reflect  the  performance  as  a  measure  of  what  the  student  has  learned  up  to  this  point  in 
training.  If  there  are  any  questions  concerning  grading  or  use  of  the  card,  please  contact  Dan  Cataneo 
at  6604  or  Lee  Lesher/DOR  2 4(>8.  Before  you  fly  with  your  student  on  a  data  ride,  please  read  this 
guide  and  the  data  cards.  If  you  are  unsure,  ask!  Thank  you  for  your  help. 
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APPENDIX  C:  SUMMARY  TASK  ITEM  ANALYSIS 


I.  Takeoff 


A  1.  Takeoff  Roll/Ground  Track  Control 

Left 

On 

Right 

1st  Observation 

4 

2 

2 

Control 

2nd  Observation 

2 

3 

3 

1st  Observation 

2 

3 

3 

Night 

2nd  Observation 

4 

4 

2 

1st  Observation 

2 

2 

3 

Day 

2nd  Observation 

3 

3 

2 

F2.  Nose  wheel  Steering 

Ove  (control 

Undeiconnol 

Appropriate 

1st  Observation 

4 

1 

3 

Control 

2nd  Observation 

0 

3 

5 

1st  Observation 

3 

3 

2 

Night 

2nd  Observation 

1 

2 

5 

1st  Observation 

3 

3 

2 

Day 

2nd  Observation 

2 

3 

3 

B.  Rotation  Speed 

Low 

On 

High 

1st  Observation 

0 

0 

8 

Control 

2nd  Observation 

1 

1 

6 

1st  Observation 

0 

2 

6 

Night 

2nd  Observation 

0 

6 

2 

1st  Observation 

1 

4 

3 

Day 

2nd  Observation 

I 

7 

0 
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Control 


1st  Observation 


3 


2nd  Observation 

4 

1st  Observation 

3 

Night 

2nd  Observation 

4 

1st  Observation 

5 

Day 

2nd  Observation 

3 

D. 

Pitch  Attitude 

Low 

1st  Observation 

5 

Control 

2nd  Observation 

4 

1st  Observation 

1 

N  ight 

2nd  Observation 

0 

1st  Observation 

0 

Day 

2nd  Observation 

1 

E. 

Runway  Alignment  after  Takeoff 

Left 

1st  Observation 

0 

Control 

2nd  Observation 

2 

1st  Observation 

0 

N  ight 

2nd  Observation 

2 

1st  Observation 

2 

Day 

2nd  Observation 

1 

II.  Straight-ln  Approach  (Either  Full  Stop  or  TG) 

A. 

Runway  Alignment  Prior  to  Descent 

Left 

1st  Observation 

5 

Control 


2nd  Observation 


3 


•’’VUTW 


1st  Observation 
2nd  Observation 
1st  Observation 
2nd  Observation 

Altitude  Prior  to  Descent 


-!00< 

-100 

-50' 

O' 

50' 

100' 

>  100 

1st  Observation 

3 

2 

1/3 

1-1/3 

1,3 

0 

I 

2nd  Observation 

3-1/2 

1-1/2 

1-1/3 

1-1/3 

1/3 

0 

0 

1st  Observation 

1 

6 

0 

0 

0 

1 

0 

2nd  Observation 

0 

2 

3 

2 

1 

0 

0 

1st  Observation 

1 

4 

1-1/3 

1/3 

1/3 

0 

1 

2nd  Observation 

1-1/2 

2-1/2 

2 

1 

0 

0 

0 

C.  Glidepath  Intercept  Point 

1st  Observation 
rol 

2nd  Observation 

1st  Observation 
t 

2nd  Observation 
1st  Observation 
2nd  Observation 

D.  Glidepath  Control 
1.  Altitude 


1st  Observation 
2nd  Observation 
1st  Observation 
2nd  Observation 
1st  Observation 
2nd  Observation 


Airspeed 


Low  On  High 


Control 

\  ight 

Day 

1st  Observation 

2nd  Observation 

1st  Observation 

2nd  Observation 

1st  Observation 

2nd  Observation 

3.  Runway  Alignment 

3 

2 

1 

3 

2-1/3 

Left 

1 

3 

3 

2 

3 

1/3 

On 

4 

3 

4 

5 

2 

4-1/3 

Right 

1st  Observation 

5 

0 

3 

Control 

2nd  Observation 

4 

2 

2 

1st  Observation 

3 

4 

1 

Night 

2nd  Observation 

0 

7 

1 

1st  Observation 

1 

6 

1 

Dav 

E. 

2nd  Observation 

Flare 

1.  Position 

1 

5 

1 

High 

On 

Low 

1st  Observation 

4 

2 

i 

Control 

2nd  Observation 

I 

2 

5 

1st  Observation 

5 

0 

2 

Night 

2nd  Observation 

2 

1 

5 

1st  Observation 

4 

1 

3 

I)av 

2nd  Observation 

5 

0 

0 
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2.  Technique 

Abrupt 

1st  Observation 

4 

Control 

2nd  Observation 

S 

1st  Observation 

4 

Mght 

2nd  Observation 

2 

1st  Observation 

2 

Day 

2nd  Observation 

1 

F. 

Touchdown 

1.  Airspeed 

Low 

1st  Observation 

2 

Control 

2nd  Observation 

6 

1st  Observation 

0 

Night 

2nd  Observation 

6 

1st  Observation 

1 

Day 

2nd  Observation 

2 

2.  Runway  Alignment 


1st  Observation 

i 

Control 

2nd  Observation 

3 

1  st  Observation 

2 

Nigh. 

2nd  Observation 

2 

1st  Observation 

0 

Day 

2nd  Observation 

0 
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111.  Touch-and-Go  (Takeoff  portion) 
A.  Takeoff  Attitude 


Low 

On 

High 

1st  Observation 

i 

0 

4 

Control 

2nd  Observation 

3 

0 

5 

1st  Observation 

1 

1 

5 

Night 

2nd  Observation 

2 

4 

1 

1st  Observation 

] 

1 

3 

Da; 

2nd  Observation 

1 

Nose  Wheel 

1 

4 

Touch 

Lift  Early 

Appropriate 

1st  Observation 

1 

2 

2 

Control 

2nd  Observation 

3-1/2 

4-1/2 

0 

1st  Observation 

1 

3 

3 

Night 

2nd  Observation 

2 

1 

4 

1st  Observation 

1-1/2 

3-1/2 

0 

Day 

2nd  Observation 

1-1/2 

3-1/2 

1 

2.  Directional  Control 

Low 

On 

High 

1st  Observation 

1 

1 

3 

Control 

2nd  Observation 

4 

4 

0 

1st  Observation 

4 

3 

0 

Night 

2nd  Observation 

0 

3 

4 

1st  Observation 

3 

2 

0 

Day 

2nd  Observation 

3 

1 

2 

Oveicontrol 

Undenrontrol 

Appropriate 

1st  Observation 

2 

2 

i 

Control 

2nd  Observation 

1 

4 

3 

1st  Observation 

1 

3 

3 

Night 

2nd  Observation 

2 

2 

3 

30 


1st  Observation 

i 

2 

2 

Day 

2nd  Observation 

i 

4 

1 

c. 

Liftoff  Speed 

Low 

On 

High 

1st  Observation 

1 

2 

1 

Control 

2nd  Observation 

4 

1 

3 

1st  Observation 

S 

0 

1 

Night 

2nd  Observation 

3 

4 

0 

1st  Observation 

3 

2 

0 

Day 

2nd  Observation 

4 

2 

0 

D. 

Pitch  Attitude 

Low 

On 

High 

1st  Observation 

0 

1 

4 

Control 

2nd  Observation 

2 

1 

5 

1st  Observation 

0 

0 

•7 

Night 

2nd  Observation 

) 

3 

3 

1st  Observation 

1 

2 

2 

Day 

2nd  Observation 

0 

2 

4 

E. 

Runway  Alignment  After  Takeoff 

Left 

On 

Right 

1st  Observation 

1 

1 

3 

Control 

2nd  Observation 

3 

5 

0 

1st  Observation 

1 

4 

2 

Night 

2nd  Observation 

1 

5 

1 

1st  Observation 

0 

4 

1 

Day 

2nd  Observation 

0 

1 

1 
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